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Before LEE E. BARRETT, ST. JOHN COURTENAY III, and 
THU A. DANG, Administrative Patent Judges. 

BARRETT, Administrative Patent Judge. 



DECISION ON APPEAL 
This is a decision on appeal under 35U.S.C. §134 (a) from the final 
rejection of claims 1-30. We have jurisdiction pursuant to 35 U.S.C. § 6(b). 
We reverse but enter a new ground of rejection. 



1 Filed January 3, 2002, titled "Method and System for Producing a 
Book from a Video Source." 

2 The two-month time period for filing an appeal or commencing a 
civil action, as recited in 37 C.F.R. § 1.304, begins to run from the decided 
date shown on this page of the decision. The time period does not run from 
the Mail Date (paper delivery) or Notification Date (electronic delivery). 
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STATEMENT OF THE CASE 

The invention 

The invention relates to a system, method, and program on a medium 
for automatically analyzing a video source to produce a book document. 
Video source data is decoded and a key frame is extracted to serve as the 
illustration part of the book. Text is extracted from the video data, for 
example, by optical character recognition of the caption data or by speech 
recognition, and used for the text part of the book. 

The claims 

Claim 1 is reproduced below: 

1 . A book producing system for producing a book, which consists of 
a text part and an illustration part, comprising: 

a video-receiving module for receiving video source data; 

a decoding module for decoding the video source data to obtain 
video data; 

a text-extracting module for extracting the text part from the 
video data according to a production guide; 

an illustration-extracting module for extracting a key frame 
from the video data according to the production guide, the key frame 
serving as the illustration part; and 

a book-producing module for producing the book according to 
the extracted text part and illustration part. 
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The references 

Myers US 2002/0037104 Al Mar. 28, 2002 

Shingo Uchihashi et al., Video Manga: Generating Semantically 
Meaningful Video Summaries, Proc. of Seventh ACM Int'l Conf. on 
Multimedia (Part 1), Oct. 1999, pp. 383-392 ("Uchihashi"). 

The rejections 

Claims 1-4, 6-14, 16-24, and 26-30 stand rejected under 35 U.S.C. 
§ 103(a) as unpatentable over Myers. 

Claims 5, 15, and 25 stand rejected under 35 U.S.C. § 103(a) as 
unpatentable over Myers and Uchihashi. 

PRINCIPLES OF LAW 
"[T]he best defense against the subtle but powerful attraction of a 
hindsight-based obviousness analysis is rigorous application of the 
requirement for a showing of the teaching or motivation to combine prior art 
references." In re Dembiczak, 175 F.3d 994, 999 (Fed. Cir. 1999). The 
motivation, suggestion or teaching may come explicitly from statements in 
the prior art, the knowledge of one of ordinary skill in the art, or, in some 
cases the nature of the problem to be solved. Id. In analyzing whether it 
would have been obvious to one of ordinary skill in the art to make a 
modification or combination, there does not have to be an express teaching, 
suggestion, or motivation (TSM) in a published article or issued patent. 
KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 419 (2007) ("The obviousness 
analysis cannot be confined by a formalistic conception of the words 
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teaching, suggestion, and motivation, or by overemphasis on the importance 
of published articles and the explicit content of issued patents."). "Even 
when obviousness is based on a single prior art reference, there must be a 
showing of a suggestion or motivation to modify the teachings of the 
reference." In re Kotzab, 111 F.3d 1365, 1370 (Fed. Cir. 2000). 

The fact that the subject matter of a reference could be modified to 
produce the claimed subject matter still requires a suggestion or motivation. 
See In re Mills, 916 F.2d 680, 682 (Fed. Cir. 1990) ("While Mathis' 
apparatus may be capable of being modified to run the way Mills' apparatus 
is claimed, there must be a suggestion or motivation in the reference to do 
so."); In re Fritch, 972 F.2d 1260, 1266 (Fed. Cir. 1992) ("The mere fact 
that the prior art may be modified in the manner suggested by the Examiner 
does not make the modification obvious unless the prior art suggested the 
desirability of the modification."). 

ISSUE 

Does Myers teach or suggest "an illustration-extracting module" and 
"a book-producing module" as recited in claim 1 and the corresponding 
limitations in claims 1 1 and 21? 

FACTS 

Myers describes "a method to portably detect and extract text 
information from captured imagery, thereby allowing new implementations 
for gathering, processing, and dissemination of information related to 
contents of captured imagery" fl| [0008]). Myers describes a text recognition 
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and extraction engine and it clearly has a "text-extracting module for 
extracting the text part from the video data," as recited in claim 1. 

Myers describes seven embodiments. For example, in a first 
embodiment, the unit could be used as a portable language translator where 
text is detected and translated to a language native to the user (]f [0010]). In 
a second embodiment, the unit is used by a visually impaired or young 
person where the image is captured and the text within the image is detected 
and audibly spoken (]f [001 1]). As another example, in a sixth embodiment, 
the unit could be deployed in a police vehicle for reading license plates, 
vehicle identification numbers (VINs) or drivers licenses and registrations 
where the text within the captured imagery is detected and used to provide 
information to the officer fl| [0015]). 

Myers describes a seventh embodiment where the device is used as a 

portable inventory assistant to capture an image having text, such as a 

product on a shelf or the odometer on a returned rental car, and the text 

within the imagery is detected and used for inventory control flf [0016]). As 

to this embodiment, Myers teaches that the text can be provided for "image 

and video indexing and archiving and inventory and shelf restocking 

control" flf [0030]). This is described as follows: 

Finally, the portable text recognition device 100 can also index 
and archive image and video, both for storage and identification, and 
as a means to increase the accuracy of targeted marketing programs. 
An example of this is to apply this technique on an internet photo 
server using the results to increase the accuracy that the pop up ads 
the user seeks is relevant. 



5 



Appeal 2008-002673 
Application 10/034,390 

11 [0078]. 

CONTENTIONS 

Appellant argues that Myers fails to teach "an illustration-extracting 
module" and "a book-producing module" as recited in claim 1 (Br. 5), and 
the corresponding limitations in claims 1 1 and 2 1 . 

The Examiner admits that Myers fails to teach "an illustration- 
extracting module" and a "book producing module for producing the book 
according to the extracted text part and illustration part," but concludes that 

it would have been obvious to one of ordinary skill in the art at the 
time of the invention to have extracted an image — illustration 
part — from a video and index the image to produce an indexed photo 
album or book, using the extracted text, because Myers teaches using 
the extracted text to index images (0030, lines 1 1-29), thus, providing 
the benefit of organizing images, and text extracted from video, in 
order to quickly, and efficiently access those images. 

Final Rej. 3; Ans. 3-4. 

Appellant argues that Myers merely discloses an apparatus and 
method for recognizing text in an image sequence and fails to teach or 
suggest anything regarding a book producing system (Br. 6). It is argued 
that Myers' description of using text to increase the accuracy of targeted 
marketing programs and pop up ads in paragraph [0078] provides no 
suggestion to use extracted text to index images in a book producing system 
(id). 

Appellant argues that Myers' teaching of indexing images in no way 
teaches or suggests extracting a key frame from the video data according to 
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the production guide to produce a book as recited in claim 1 and it would be 

impermissible hindsight to incorporate the teachings of Myers with 

teachings not found in the utilized prior art (Br. 6-7). 

The Examiner responds: 

Regarding claims 1,11, and 21, the Appellant notes there is no 
motivation to modify the apparatus, which uses text to index images 
into a book producing system (page 6, parag. 2). The Examiner 
disagrees, because Myers teaches producing books using text 
extracted from a video (0030, 0035-0036). Myers fails to explicitly 
teach extracting, and producing a book using an extracted illustration 
part. However, it would have been obvious to extract the illustration 
part, besides the text, to produce a book, because Myers teaches the 
indexing of images using the extracted text (0030, lines 1 1-29). This 
would provide the benefit of producing a book, which contains both 
images, and text describing/organizing such images to provide a 
quick, and efficient access of those images. 

Ans. 9. 

ANALYSIS 

As to Appellant's argument that Myers fails to teach a book-producing 
module, the Examiner finds that "Myers teaches producing books using text 
extracted from a video (0030, 0035-0036)" (Ans. 9). Paragraphs [0035] and 
[0036] of Myers teach extracting text from video, which is not disputed. 
Paragraph [0030] of Myers states that the text can be provided for "image 
and video indexing and archiving and inventory and shelf restocking 
control," but the Examiner does not explain how "indexing and archiving" 
can be interpreted as producing a book. The Examiner has provided no 
definition of a "book" that is broad enough to read on mere indexing or any 
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logical reasoning of why indexing is a book. Based on the evidence before 
us, we find that Myers does not teach a book-producing module. 

As to Appellant's argument that Myers fails to teach an illustration- 
extracting module, the Examiner concludes that it would have been obvious 
to extract illustrations to produce a book because Myers teaches indexing of 
images using the extracted text. Myers teaches that the text can be provided 
for "image and video indexing and archiving and inventory and shelf 
restocking control" [0030]) and that "the portable text recognition 
device 100 can also index and archive image and video, both for storage and 
identification" flf [0078]). It is not clear to us, and the Examiner does not 
explain, how this teaches extracting a key frame (or any frame) from a video 
to serve as the illustration part of a book (or other structure representative of 
the video). Myers appears to state that text could be recognized in a video 
for the purpose of indexing and archiving the video, not for the purpose of 
extracting key frames to serve as representative frames of the video. 

CONCLUSION 

Myers does not teach or suggest "an illustration-extracting module" 
and "a book-producing module" as recited in claim 1 and the corresponding 
limitations in claims 1 1 and 21. The rejection of claims 1-4, 6-14, 16-24, 
and 26-30 under § 103(a) over Myers is reversed. 

The rejection of dependent claims 5, 15, and 25 over Myers and 
Uchihashi is reversed because the rejection of the independent claims has 
been reversed. We note that Uchihashi, as applied to dependent claims 5, 
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15, and 25, cures the deficiencies of Myers as to the independent claims. 
However, we have designated this a new ground of rejection infra because 
Uchihashi is not argued in the Appeal Brief and is not relied upon by the 
Examiner to meet the independent claims. 

NEW GROUND OF REJECTION 

Independent claims 1,11, and 21 are rejected under 35 U.S.C. 
§ 103(a) as unpatentable over Uchihashi and Myers. We leave it to the 
Examiner to address the patentability of the dependent claims. 

Uchihashi discusses "methods for automatically creating pictorial 
video summaries that resemble comic books" (holding omitted) (Abstract). 
Uchihashi discusses that "[w]hile any collection of video keyframes can be 
considered a summary, our goal is to produce a meaningful and concise 
representation of the video" (p. 383, right col.) by "choosing only the most 
salient images and efficiently packing them into a pictorial summary" (id.). 
Uchihashi discusses selecting and preparing keyframes (pp. 383-385), which 
implicitly requires a video-receiving module, a decoding module and an 
illustration-extracting module. 

Uchihashi describes that a "two-dimensional layout is most 
appropriate for a printed synopsis, such as a 'comic book' format" (p. 385, 
right col.). Thus, Uchihashi teaches a book producing module. We note that 
the claims do not recite any specific book format. 

Uchihashi describes that "[fjextual annotations enhance the quality of 
the video summary" (p. 388, right col.) and "video summarization methods 
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use text, typically from closed-caption subtitles or manual transcriptions" 
(id.). One of ordinary skill in the art would find this to teach or at least 
suggest a text-extracting module for extracting the text part (caption data) 
from the video. Uchihashi further describes that "the current state of the art 
of speech recognition is not robust enough to be practical at present" (p. 389, 
left col.). Thus, while Uchihashi does not use speech recognition to extract 
the text part, one of ordinary skill in the art would have been taught that 
speech recognition could be used to extract text if robustness was not critical 
or when the state of the art improves. 

Because we explain why one of ordinary skill in the art would have 
recognized certain limitations to be present in Uchihashi, we enter the 
rejection under § 103(a) rather than § 102. Although Uchihashi alone is 
considered sufficient to establish obviousness of the subject matter of the 
independent claims, Myers is cited as evidence of another method of 
extracting text data from video data using optical character recognition. 

CONCLUSION 

The rejections of claims 1-30 are reversed. 

A new ground of rejection is entered as to claims 1,11, and 21. 

This decision contains new grounds of rejection pursuant to 37 C.F.R. 
§ 41.50(b). 37 C.F.R. § 41.50(b) provides that "[a] new ground of rejection 
pursuant to this paragraph shall not be considered final for judicial review." 

37 C.F.R. § 41.50(b) also provides that the appellant, WITHIN TWO 
MONTHS FROM THE DATE OF THE DECISION, must exercise one of 
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the following two options with respect to the new ground of rejection to 
avoid termination of the appeal as to the rejected claims: 

(1) Reopen prosecution. Submit an appropriate amendment of 
the claims so rejected or new evidence relating to the claims so 
rejected, or both, and have the matter reconsidered by the examiner, in 
which event the proceeding will be remanded to the examiner. . . . 

(2) Request rehearing. Request that the proceeding be reheard 
under § 41.52 by the Board upon the same record. . . . 

Requests for extensions of time are governed by 37 C.F.R. § 1.136(b). 
See 37 C.F.R. §4 1.50(f). 



REVERSED - 37 C.F.R. § 41.500)) 



msc 
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